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Application No. 



Applicant(s) 



Interview Summary 



10/814,851 



COHEN ET AL 



Examiner 



Art Unit 



Nancy Bitar 



2624 



All participants (applicant, applicant's representative, PTO personnel): 



(1) Nancy Bitar . 



(3). 



(2) Jason Lindh . 



(4). 



Date of Interview: 10 October 2007 . 

Type: a)£3 Telephonic b)D Video Conference 

c)D Personal [copy given to: 1)D applicant 2)D applicant's representative] 

Exhibit shown or demonstration conducted: d)D Yes e)IEl No. 
If Yes, brief description: . 

Claim(s) discussed: 1 and 2 . 

Identification of prior art discussed: Mittal et al ( US 2005/0286764) and DeMenthon et al (EAR99 -05844) . 
Agreement with respect to the claims f)D was reached. g)[x] was not reached. h)D N/A. 



Substance of Interview including description of the general nature of what was agreed to if an agreement was 
reached, or any other comments: See Continuation Sheet . 

(A fuller description, if necessary, and a copy of the amendments which the examiner agreed would render the claims 
allowable, if available, must be attached. Also, where no copy of the amendments that would render the claims 
allowable is available, a summary thereof must be attached.) 

THE FORMAL WRITTEN REPLY TO THE LAST OFFICE ACTION MUST INCLUDE THE SUBSTANCE OF THE 
INTERVIEW. (See MPEP Section 713.04). If a reply to the last Office action has already been filed, APPLICANT IS 
GIVEN A NON-EXTENDABLE PERIOD OF THE LONGER OF ONE MONTH OR THIRTY DAYS FROM THIS 
INTERVIEW DATE, OR THE MAILING DATE OF THIS INTERVIEW SUMMARY FORM, WHICHEVER IS LATER, TO 
FILE A STATEMENT OF THE SUBSTANCE OF THE INTERVIEW. See Summary of Record of Interview 
requirements on reverse side or on attached sheet. 




Examiner Note: You must sign this forry unless it is an 
Attachment to a signed Office action. 
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Continuation Sheet (PTOL-413) 



Application No. 10/814,851 



Continuation of Substance of Interview including description of the general nature of what was agreed to if an 
agreement was reached, or any other comments: Applicant argued that figure 9 of Mittal et al does not teach 
segmenting 3D based on color as pointed out by Examiner it is true that the primary reference does not explicitly 
teaches the color segmentation but the secondary reference teaches color segmentation so the 103 rejection still 
meets claims 1-19.Applicant argued that pointing out the second reference will be a new ground of argument 
preventing finality of the next action. Examiner disagrees with applicant since it will not form new ground of rejection. 
(See attached informal communication presented by applicant on 9/11/2003). 
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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

Serial No. .. 10/814,851 

Filing Date Mar 31, 2004 

First Named Inventor Michael F. Cohen 

Assignee Microsoft Corporation 

Group Art Unit 2624 

Examiner Nancy Bitar 

Attorney's Docket No MS1-1895US 

Title STYUZATION OF VIDEO 

Informal Communication in Preparation for 
Scheduling an Interview 



To: Examiner Bitar 

Fax: 571-270-2041 



From: Jason F. Lindh 

Lee & Hayes, PLLC 

421 W. Riverside Avenue, Suite 500 

Spokane, WA 99201 

jason@leehayes.com 

(Tel. 509-324-9256; Fax 509-323-8979) 



Dear Examiner Bitar: 

[0001] This communication provides an agenda for a phone interview 
of this matter. My assistant will be contacting you to schedule an interview. 
If you would prefer to schedule the interview, then please contact my 
assistant or me directly. Our contact info is on the signature page of this 
document. Thank you in advance for talking with me about this matter. 
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Interview Agenda: 

■ Restriction Requirement 

■ Discussion of proposed amendments 

Restriction Requirements 

[0002] Applicant proposes affirming the election in the formal response 
to the outstanding office action. 

Proposed Amendments 

[0003] Claims 1-19 have been rejected under 35 U.S.C. 103(a) as 
being unpatentable over U.S. Patent Publication No. 2005/0286764 to Mittal 
in light of DeMenthon (Spatio-Temporal Segmentation of Video by 
Hierarchical Mean Shift Annalysis, July 2002). Applicant respectfully 
traverses these rejections but proposes the following amendments to 
advance prosecution of the application. 

[0004] Independent claim 1 recites a method of stylizing video, 
comprising: 

performing a spatio-temporal segmentation 
analysis on the video to identify three 
dimensional volumes of contiguous pixels having 
[a] similar [ character i stic ] color : 

receiving an input identifying a group of 
the three dimensional volumes; and 

identifying the group of three dimensional 
volumes as a single semantic region. 
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[0005] The cited art does not teach nor suggest performing a spatio- 
temporal segmentation analysis on the video to identify three dimensional 
volumes of contiguous pixels having similar colors. The Examiner points to 
Mittal as teaching this element with reference to dependent claim 2, which 
has been incorporated into claim 1 to further distinguish claim 1 from the 
cited art. The Examiner specifically points to Figure 9 of Mittal as teaching 
the color representation. Applicant respectfully traverses this analysis. 

[0006] Applicant would initially point out that that figure 9 is illegible in 
its present state. Figure 9 supposedly shows an embodiment of Mittal 
having input frames and detection components depicted in at least three 
colors (pink, green and white). (See Paragraph [0039]). Because the 
drawings are in black and white, it is impossible to determine colors. 
Further, because of the quality of the images in the electronic file, it is 
impossible to determine if Figures 9A and 9B depict anything other than 
misshapen blobs. 

[0007] Additionally, independent claim one requires that "performing a 
spatio-temporal segmentation analysis on the video to identify three 
dimensional volumes of contiguous pixels having [a] similar [ character i stic ] 
color ". Mittal does not teach this element. Instead, Figure 9 of represents 
results from a simulated sequence wherein residuals rl and r2 are 
represented by the colors green and pink respectively. These are arbitrary 



Serial No.: 10/814,851 . 

Atty Docket No.: MS1-1895US .3. lee^hayeS The Business of) p" 

Atty/Agent: Jason F. Lindh ^^^m sotrntaa 

Informal Communication in Preparation for Scheduling 



3/10 - RCVD AT 9/11/2007 9:57:15 AM [Eastern Daylight Time] * SVR:USPTO-EFXRF-5/7 * DNIS:2702041 * CSID:15092672653 * DURATION (mm-ss):05-28 



To: Page.4of10 



2007-09-1 1 13:57:34 (GMT) 15092672653 From: Lee & Hayes 

INFORMAL COMMUNICATION: Please do not put in the file 



colors chosen to represent the residuals of the detection measures. Any 
segmentation was not based upon colors. 



[0008] Mittars invariant transformation is determined by measuring 
image pixel intensity that is transformed and evaluated in illumination- 
invariant space. (See at least paragraphs [0019], [0021], [0024]-[0025]). 
The intensity of an image pixel is different than the pixel color. As such, 
Mittal cannot teach each and every element of independent claim 1. 



[0009] Independent claim 13 has been amended to include a similar 
element. As such, independent claim 13 would be allowable for the 
aforementioned reasons as shown above with relation to independent 
claim 1. 



Conclusion 



[0010] I look forward to talking to you about the proposed 
amendments. 



Respectfully Submitted, 



Dated: September 11, 2007 By: 

Jason F. Lindh 
Reg. No. 59090 
(509) 324-9256 x215 
jason@leehayes.com 
www.leehayes.com 



My Assistant: Carly Bokarica 
(509) 324-9256 x264 
carly@leehayes.com 
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Appendix of Claims with Proposed Amendments 



1. (Currently Amended) A method for stylizing video, 
comprising: 

performing a spatio-temporal segmentation analysis on the 
video to identify three dimensional volumes of contiguous pixels having [a] 
similar [ character i st i c ] color ; 

receiving an input identifying a group of the three dimensional 
volumes; and 

identifying the group of three dimensional volumes as a single 
semantic region. 

2. (Cancel) 

3. (Original) The method of claim 1, wherein the spatio- 
temporal segmentation analysis comprises an anisotropic kernel mean shift 
segmentation procedure. 

4. (Original) The method of claim 1, wherein the input 
comprises an interactive user input. 

5. (Original) The method of claim 1, wherein the three 
dimensional volumes of contiguous pixels comprise segments. 
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6. (Original) The method of claim 5, wherein the user 
input comprises outlining a plurality of segments. 

7. (Original) The method of claim 6, wherein the 
outlining is performed on a number of keyframes of the video, the number 
of keyframes being fewer than a total number of frames of the video. 

8. (Original) The method of claim 7, wherein additional 
segments on frames of the video other than keyframes are identified by 
determining a relationship of the additional segments to the segments 
outlined on the keyframes. 

9. (Original) The method of claim 8, wherein the 
relationship comprises at least a portion of the additional segments being 
enclosed by one or more of the segments outlined on the keyframes. 

10. (Original) The method of claim 9, wherein the at least 
a portion comprises at least a majority of pixels of the additional 
segments. 

11. (Original) The method of claim 1, further comprising 
applying a stylization to the single semantic region. 

12. (Original) The method of claim 11, wherein the 
stylization comprises a mean shift technique. 
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13. (Currently Amended) A computer-readable medium 
having computer-executable instructions for stylizing video, the 
instructions comprising: 

performing a spatio-temporal segmentation analysis on the 
video to identify three dimensional volumes of contiguous pixels having [a] 
similar [ ch a ract e rist i c ] color ; 

receiving an input identifying a group of the three dimensional 
volumes; and 

identifying the group of three dimensional volumes as a single 
semantic region. 

14. (Original) The computer-readable medium of claim 

13, wherein the instructions further comprise deriving a set of edge sheets 
that represent the surface of the single semantic region and associating 
the edge sheets with the semantic region. 

15. (Original) The computer-readable medium of claim 

14, further comprising rendering the edge sheets as a curve between the 
semantic region and another portion of the video. 

16. (Original) The computer-readable medium of claim 
14, wherein a thickness of the edge sheets is determined based on criteria 
associated with the single semantic region. 
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17. (Original) The computer-readable medium of claim 
16, wherein the criteria comprises a position of the edge sheet relative to 
an arclength of the edge sheet. 

18. (Original) The computer-readable medium of claim 
16, wherein the criteria comprises a duration of existence of the semantic 
region in the video. 

19. (Original) The computer-readable medium of claim 
16, wherein the criteria comprises a movement of the semantic region in 
the video. 
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20. (Original) A computer-readable medium having 
computer-executable instructions for stylizing video, the instructions 
comprising: 

determining a set of volumetric objects by mean shift video 
segmentation, each volumetric object being a segment; 

indicating on a limited number of keyframes how segments 
should be merged into a semantic region; and 

interpolating the indications between keyframes by a mean 
shift constrained interpolation technique to propagate the indication to 
frames between keyframes. 

21. (Original) The computer-readable medium of claim 

20, further comprising drawing paint strokes within the semantic region at 
keyframes. 

22. (Original) The computer-readable medium of claim 

21, wherein the paint strokes are similarly interpolated. 

23. (Original) The computer-readable medium of claim 
20, wherein a surface of the semantic region is smoothed. 

24. (Original) The computer-readable medium of claim 
20, wherein an edge sheet is determined that corresponds to a surface of 
the semantic region. 
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25. (Original) The computer-readable medium of claim 
20, wherein a stroke sheet is determined that corresponds to a surface 
lying within the semantic region 
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